A QUASI - NEWTON ACCELERATION OF THE EM ALGORITHM Kenneth
نویسندگان
چکیده
The EM algorithm is one of the most commonly used methods of maximum likelihood estimation. In many practical applications, it converges at a frustratingly slow linear rate. The current paper considers an acceleration of the EM algorithm based on classical quasi-Newton optimization techniques. This acceleration seeks to steer the EM algorithm gradually toward the Newton-Raphson algorithm, which has a quadratic rate of convergence. The fundamental di erence between the current algorithm and a naive quasi-Newton algorithm is that the early stages of the current algorithm resemble the EM algorithm rather than steepest ascent. Numerical examples involving the Dirichlet distribution, a mixture of Poisson distributions, and a repeated measures model illustrate the potential of the current algorithm.
منابع مشابه
A quasi-Newton acceleration for high-dimensional optimization algorithms
In many statistical problems, maximum likelihood estimation by an EM or MM algorithm suffers from excruciatingly slow convergence. This tendency limits the application of these algorithms to modern high-dimensional problems in data mining, genomics, and imaging. Unfortunately, most existing acceleration techniques are ill-suited to complicated models involving large numbers of parameters. The s...
متن کاملExpectation Maximization for Combined Phylogenetic and Hidden Markov Models
An expectation maximization (EM) algorithm is derived to estimate the parameters of a phylogenetic model, a probabilistic model of molecular evolution that considers the phylogeny, or evolutionary tree, by which a set of present-day organisms are related. The EM algorithm is then extended for use with a combined phylogenetic and hidden Markov model. An efficient method is also shown for computi...
متن کاملQuasi-Newton Methods for Nonconvex Constrained Multiobjective Optimization
Here, a quasi-Newton algorithm for constrained multiobjective optimization is proposed. Under suitable assumptions, global convergence of the algorithm is established.
متن کاملOn Aitken's Method and Other Approaches for Accelerating Convergence of the Em Algorithm
The EM algorithm of Dempster, Laird, and Rubin (1977) is a broadly applicable approach that has been widely applied to the iterative computation of maximum likelihood estimates in a variety of incomplete-data problems. A criticism that has been levelled at the EM algorithm is that its convergence can be quite slow. Unfortunately, methods to accelerate the EM algorithm do tend to sacriice the si...
متن کاملA class of multi-agent discrete hybrid non linearizable systems: Optimal controller design based on quasi-Newton algorithm for a class of sign-undefinite hessian cost functions
In the present paper, a class of hybrid, nonlinear and non linearizable dynamic systems is considered. The noted dynamic system is generalized to a multi-agent configuration. The interaction of agents is presented based on graph theory and finally, an interaction tensor defines the multi-agent system in leader-follower consensus in order to design a desirable controller for the noted system. A...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1999